The human visual system is typically able to adapt to lighting variations across scenes, perceiving details in regions with very different dynamic ranges. Most image recording systems, however, fail this dynamic range compression task. As a result, images produced by these image recording systems are often of poor quality, compared to images produced by human perception. Another task that is often poorly performed by the image recording systems is that of color constancy. Humans perceive color in a way that is fairly independent of scene illumination, whereas the image recording systems are strongly influenced by spectral shifts.
The above problems can be stated mathematically by describing a relationship between an acquired image S, a reflectance of objects of the image R, and an illumination L in a pixel-wise multiplication, or:S=L·R.This expression means that at each point in the image S, the color value is the multiplication of the reflectance value by the illumination value. Given an image S, the problem to be solved is removal of the effects of illumination and recovery of the reflectance image. That is, given S, find both R and L. However, calculation of both R and L is typically not possible. The solution then, is to generate a methodology that can estimate R and L.
Retinex theory deals with compensation for illumination in images. The goal of the Retinex theory is to decompose a given image S into the reflectance image R, and the illumination image L, such that at each point (x,y) in the image domain, S(x,y) equals the product of R(x,y) and L(x,y). The benefits of such decomposition include the possibility of removing illumination effects of back/front lighting, enhancing photographs and other image capture methods that include spatially varying illumination, such as images that contain indoor and outdoor zones, and correcting colors in images by removing illumination-induced color shifts.
As noted above, recovering the illumination L from a given image S is known to be a mathematically ill-posed problem, and known algorithms vary in the manner and effectiveness of overcoming this limitation. The Retinex approach provides the framework for one such method. The Retinex methodology was motivated by Edward Land's research of the human visual system, which is described in R. H. Land, “The Retinex Theory of Color Vision,” Sci. Amer., Vol. 237, pp. 108-128 (1977).
The first Retinex algorithms were of the random walk type. Subsequent Retinex algorithms used homomorphic filtering. Yet another group of Retinex algorithms are based on solving a Poisson equation.
FIG. 1 is a block diagram of an algorithm 10 that is representative of the general class of prior-art Retinex algorithms. In FIG. 1, an input image S 11 is applied to a logarithmetic module 12 to produce a logarithmetic version 13 of the input image S and its two components, illumination L and reflectance R. That is, s=log S, l=log L, and r=log R, and thereby, s=l+r. Using the logarithmetic version 13 of the input images, an estimator module 14 computes an estimate 17 of the illumination, designated in FIG. 1 as l*. The estimate 17 (l*) is then combined with the logarithmetic version 13 (s) at summer 18 to produce an estimate 19 of the reflectance (designated r*). Finally, the estimate 19 (r*) is converted from a logarithm to a number (antilogarithm) corresponding to the logarithm at expander 20, to produce a real number value as an estimate 21 of the reflectance (designated as R*). Prior art Retinex algorithms usually employ the same process as shown in FIG. 1.
The Retinex-based algorithms take several different forms. One such form is the random walk algorithm, which is a discrete time random process in which the “next pixel position” is chosen randomly from neighbors of the current pixel position. Random walk type Retinex algorithms are variants of the following basic formulation: A large number of walkers are initiated at random locations of the logarithmetic version 13 (s), adopting a gray-value of their initial position. An accumulator image A that has the same size as s is initialized to zero. As a walker walks around, the walker updates A by adding the value of the walker to each position (x,y) that the walker visits. The illumination image is obtained by normalizing the accumulator image A, i.e., the value at each position of the accumulator image A is divided by the number of walkers that visited that position.
By using many walkers with long paths, one can easily verify that the accumulator value at each position converges to a Gaussian average of that position's neighbors.
Another type of Retinex algorithm uses homomorphic filtering, where a low-pass filter is used to reconstruct l from s. Homomorphic Retinex algorithms are based on the fact that the reflectance image R corresponds to the sharp details in the image (i.e., the edges), whereas the illumination L is expected to be spatially smooth. Then, a reasonable guess for l is l*=LP{s}, where LP is a convolution with a wide Gaussian kernel. Thus, the Retinex algorithm using homomorphic filtering actually applies the same process as the random walk algorithms by a single direct convolution.
Since the illumination L is expected to be spatially smooth, the derivative of the illumination L should be close to zero. However, considering the assumption that the reflectance R is piece-wise constant, the derivative of the reflectance R should vanish almost everywhere, with high values along edges of an image. Taking the derivative of the sum s=l+r and clipping out high derivative peaks, implies that the clipped derivative signal corresponds only to the illumination L. Poisson equation-type Retinex algorithms that rely on the Mondrian world model, use the above assumptions on the reflectance R as a piece-wise constant image. Applying the Laplacian, and the following clipping operation:τ(Δs)=Δs where |Δs|<ττ(Δs)=0 otherwise,yields the following Poisson equationΔl*=τ(Δs)A solution to the Poisson equation may involve an iterative procedure that effectively inverts the Laplacian operator. Improvements to the method involve extracting discontinuities from the image gradient magnitude instead of the Laplacian to provide better boundary conditions.
One solution involves use of an iterative algorithm having a “reset” non-linearity that enforces the constraint l≧s. The algorithm performs the interactive procedure.       l          n      +      1        *    =      max    ⁢          {                                                  l              n              *                        +            s                    2                ,                                            l              n              *                        +                                          D                n                            ⁡                              [                                  l                  n                  *                                ]                                              2                    }      where Dn is a translation operator, shifting the image s by the nth element of a sequence of spirally decaying translation vectors. Removing the max operation yields a simplified version       l          n      +      1        *    =                    l        n        *            +                        D          n                ⁡                  [                      l            n            *                    ]                      2  The above equation is a simple averaging operation that smoothes the images. The non-linear (max) operation forces the illumination image l to satisfy the constraint l*≧s.
Despite use of these algorithms, current image recoding and image enhancement systems and methods cannot produce images that are of sufficient quality to be comparable to images as perceived by the human vision system. Thus, an improved method and system are required to better remove illumination effects and to recover the reflectance image.